Method for detecting and representing one or more objects, for example teeth

ABSTRACT

A method for detecting and representing one or more objects, such as teeth, their preparations and their immediate environment, using a camera. A first recording is made wherein a still image is produced. The still image is blended into a current, mobile search image in at least one sub-area in the second step, so that both images are recognizable. In the third step, the camera is positioned in such a way that the search image overlaps the blended-in still image in at least one sub-area. The second recording process is initiated in a fourth step.

BACKGROUND OF THE INVENTION

The invention relates to a method for detecting and representing one ormore objects, for example teeth or their preparations and theirimmediate vicinity, using a camera for obtaining three-dimensional data.These data can be used for producing a fitting.

DESCRIPTION OF THE RELATED ART

EP 0 250 993 B1 discloses a method for spatially recording andrepresenting teeth, their preparations and their immediate vicinityusing a camera for obtaining three-dimensional data. The data is usedfor producing a fitting. In order to determine the suitable recordingposition of the camera, a reference pattern is projected to obtain datadepth. First, however, a moving video search image is generated anddisplayed on a monitor on which the projected reference pattern does notappear. If the camera is in the desired recording position, then inresponse to an initiation command, an image recording sequence willproceed for generating and storing a data record. The data recordcontains image information items corresponding to the last search imagewhich can be displayed in the form of a contrast image as drawingoriginal on the monitor. The data record also contains depth valuescongruent with the image information, so that a construction performedon the drawing original together with the depth values defines thefitting three-dimensionally. For the construction of the fitting, thedentist inputs boundary lines using a drawing aid. An appropriatedrawing aid in this case is a mouse.

In the process of producing fittings for tooth reconstruction, thespatial recording is embedded in a superordinate method which allows fordifferent recordings to be related to one another. Such differentrecordings have been generated from various positions of the 3D camerawith respect, i.e., for the transformation parameters which transformall the recordings into a common reference system (translation,rotation) to be determined. This correlation or, alternatively,registration of the recordings with respect to one another is achieved,in principle, by employing software methods which rotate and translatean image data record until, in the areas in which it represents the sameobject as another image data record with which it is intended to becorrelated, it corresponds optimally to said other image data record.However, software methods of this type function only when the relativeposition of the images is adequately known as a result of previousmeasures, i.e., the images must be coarsely correlated with one another.The production of such coarse correlation is achieved in the prior artby clearly defined contours of the object (e.g. periphery) being markedin a first image through the use of a drawing aid (mouse).

When the second 3D measurement recording is created, these markings areinserted into the video search image and the measuring camera is movedinto a position such that the markings correspond as well as possible tothe corresponding structures on the video search image. A secondmeasurement is then carried out in this position.

The interactive definition of the contour lines makes it necessary forthe operator to continually alternate between the camera and the drawingaid during the sequence of a plurality of recordings. This isdisadvantageous, particularly when different recordings of the sametooth and of its surroundings have to be correlated with one another, orwhen rows of teeth have to be recorded over a plurality of recordings.

BRIEF DESCRIPTION OF THE INVENTION

It is an object of the invention to provide a method for detecting andrepresenting one or more objects in which different individual images ofthe same object or of adjacent objects are related to one anotherwithout using mechanical apparatuses, such as drawing aids, in theprocess.

According to the invention, after the production of the first recording,in a second step, the still image is inserted into a current, movingsearch image, at least in a sub-area, so that both images arerecognizable. In a third step, the camera is positioned in such a waythat the search image attains congruence with the inserted still imageat least in a sub-area. And in a fourth step, the second recording isinitiated.

In the interaction of the hand orienting the camera and the softwareprocessing the images, the present method enables the manual coarsecorrelation of two or more image data records that represent the sameobject in sub-areas.

The recording advantageously yields a 3D data record, that is the imagedata record also comprises depth values. The coarsely correlated 3D datarecords can then be correlated precisely by means of automaticcomputation methods as in the prior art. That is, the transformationparameters between the reference systems of the two individualrecordings are automatically determined computationally.

In this case, the requirements made of the coarse correlation depend onthe configuration of the computational correlation method and theavailable computing power and time. Generally it holds true that thebetter the coarse correlation, the faster and the more accurately thecomputational methods function for exact correlation. In the context ofthe superordinate method for producing dentures, the need arises tocorrelate a plurality of recordings with one another, to be precise inthree variants.

In order to measure an object from different directions or distances,and to be able to detect undercuts and/or to extend the depthmeasurement range and/or to increase the accuracy by averaging, thefirst recording is carried out by applying the method according to theinvention. What is achieved is that during the second and, ifappropriate, further recording, only the viewing angle and the distancefrom the object are varied but the object always appears inapproximately the same position on the image. This produces anadvantageous starting position for the automatic, exactly calculatedcorrelation.

In order to measure an object in the original state and after it hasbeen altered in sub-areas, such as a row of teeth before and/or afterthe preparation of an individual tooth, the first measurement of theobject in the original state is carried out and. By applying the methodaccording to the invention, the second recording (of the changed object)is effected from as far as possible the same position of the camera asthe first recording and a coarse correlation is thus achieved. Thesubsequent automatic exact correlation is effected using the objectareas which were not altered. This procedure can also be used forprogressive documentation.

By virtue of the fact that the object to be recorded in the second imagehas been altered relative to the object recorded in the first image, thesurroundings of the altered object having remained essentiallyunchanged. This process makes it possible to document restorations andto detect changes to the restorations.

If the intention is to measure objects which are more extensive than themeasurement zone of the camera, part of the object can be measured in afirst position of the camera. Afterward, a second measurement recordingis created, which covers part of the area of the first measurementrecording but also captures a new part of the object. In this case, thestill image is displaced by a predetermined distance and/or apredetermined angle and is thus superposed on the moving search imageonly in a sub-area. Using the area of overlap, the two measurementrecordings are coarsely correlated by the method according to theinvention, and exactly correlated according to automatic methods knownper se. As a result, it is possible for successively recorded sub-imagesof a complete jaw to be strung together exactly and thus for a dentalarch as a whole to be detected and measured.

By virtue of the fact that the first image and the second image aredisplaced relative to one another, the displacement amounting to atleast 1/10 and advantageously ¼ of the extent of the image in thedirection of the displacement, it is possible to detect one or moreobjects which exceed the size of the actual image zone. The correlationis effected by ascertaining identical areas and transforming thethree-dimensional data obtained into a single measurement model.

The displacement and/or rotation are advantageously effected in a mannerdependent on the area to be recorded using knowledge about the imageinformation to be expected from the object to be recorded for the secondimage. This knowledge may be information determined from statisticalevaluations or from individual peculiarities typical of the object. Thismeans that manual displacement is unnecessary, as a result of which theoperability is improved.

The above-described insertion of the first image can be done by thefirst image being superposed semi-transparently on the video searchimage during the second recording. Semi-transparently means that bothimage information items are superposed in such a way that they bothappear superposed on the screen and can be recognized by the eye.

The representation of the image information items of the first image andof the moving video search image can also be produced by additivemixing.

It is likewise possible for the first image and the moving video searchimage to be represented in such a way that the image information itemsalternate row by row or column by column. The image information itemsmay also be interwoven like a chessboard, i.e., the first image isrepresented on the “white zones” and the moving video search image isrepresented on the “black zones”.

The combination of the image information items takes place only in theareas where both images yield information items. The image isexclusively represented at the locations where only one image yieldsinformation. As a result of this, only the areas that are crucial forassessing the positioning are represented, without disturbingsuperpositions taking place.

A further improvement of the manual alignment of the video search imagewith the first image is produced by the first image being represented inone hue, in particular in red, whereas the search image is representedin black and white or in another color.

It may be advantageous if the first image is not superposed directly,but rather as modifications thereof. Such modifications generated byimage-processing means allow for the recognition accuracy to beimproved.

Other objects, advantages and novel features of the invention willbecome more apparent from the following detailed description of theinvention when taken in conjunction with the accompanying drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

The method according to the invention is explained with reference to thedrawing, in which:

FIG. 1 is a flow diagram of the method sequence for measuring a dentalarch of a first and of a second recording;

FIG. 2 is a top plan view of the already created first recording of anobject;

FIG. 3 is a top plan view of the longitudinally displaced firstrecording prior to the insertion of the video search image;

FIG. 4 is a top plan view of the video search image with the overlappedfirst recording;

FIG. 5 is a diagram showing additive mixing of the still image with thesearch image;

FIG. 6 is a diagram showing the row-by-row intermeshing of the stillimage with the search image, and

FIG. 7 is a diagram showing the intermeshing of the still image with thesearch image point by point like in a chessboard.

DETAILED DESCRIPTION OF THE INVENTION

The first recording, already described in EP 0 250 993 B1, is initiatedby a search phase preceding the actual measurement operation. Thisprocess is shown in accordance with the flow diagram from FIG. 1. Inthis search phase 1, a measurement camera initially operates like aconventional video system. The tooth part which appears in the recordingwindow is registered by an imaging optical system and sensor, and isrepresented on a monitor as a standard television image 2. The observedtooth is illuminated by the camera for the purpose of spatialmeasurement, to be precise with a grid-like reference pattern. However,this reference pattern should not also appear on the search image, sincethe actual image content is thereby superposed. Therefore, the patternis eliminated. The camera is oriented using the video search image insuch a way that the measurements required for the measurement operationcan be carried out successfully. If the production of a fitting for atooth reconstruction is involved, then the positioning of the camera inthe search phase 1 is to be chosen in such a way that it corresponds tothe later entry axis of the fitting.

In the next step 3, the recording is initiated after the suitableposition has been found. During the measurement operation, the referencepattern is projected onto the tooth surface in different spatialpositions. The resulting images are stored in a memory. Altogether, thistakes less than ⅕ of a second. Afterward, the stored information istransformed and stored as relief 4 in the form of depth data for eachpixel. In addition, a contrast image 5 is generated which can bedirectly represented on the monitor. In its pseudo-plastic nature, thisimage 1 is like the video search image and thereby allows the dentist toeffect immediate monitoring 6 of the recording.

According to the invention, a second recording 7 is then prepared. Afterthe generation of the first recording, the contrast image 5 is insertedas a still image on the monitor in the region of the current videosearch image. During the orientation of the camera, at least a part 5′of the first contrast image and also the actual video search image 8 aredisplayed on the monitor in the same window. Using the structuresrepresented in the first contrast image 5′, it is possible to orient thecurrent video search image 8 in such a way that sub-areas of the imageare brought into congruence.

In this position, the second recording 9 is initiated and the recordingis represented in the same window 9′ of the monitor without the firstcontrast image.

If the monitoring 15 of the recording is satisfactory, then, in afurther step 16, a computational, automatic correlation of the depthvalues for the overlapping areas is performed. This step uses the dataof the first and second recordings that are related to one another. Thesuperposed correlated relief is stored in the memory.

FIG. 1 illustrates a method sequence in which the second recording iscarried out, relative to the first recording, with an object displacedby about half of the first image. This is explained in more detail withregard to FIGS. 2 to 4.

Teeth, 10, 11, 12, of a row of teeth are represented as objects to bemeasured in FIG. 2. The tooth 10 and approximately half of the tooth 11being represented by means of a camera as video search image within awindow 13 on a monitor (not illustrated). The area represented in thewindow 13 is measured by means of the camera in a first recording afterthe latter has been oriented using the video search image in accordancewith the requirements for the measurement operation. From the datarecord generated by the camera, a contrast image is produced and depthdata are generated. In terms of its information content, the contrastimage corresponds approximately to the video search image, so that theoperator can check the quality of the recording. This contrast image isrepresented in the window 13 after the recording.

In order to measure the tooth 11, the second recording is then preparedby the contrast image of FIG. 2 being displaced within the window 13 byapproximately half of the image length away from the object 11 to bemeasured. Thus an area 21 is produced in which there is initially noimage information present, represented in FIG. 3.

In part of the contrast image fo the first recording, the camera isaligned with the tooth 11 that is to be measured. Also, the video searchimage of the second recording to be prepared is displayed in the upperarea 22 of the window 13. By contrast, the current moving video searchimage is represented in the area 21 of the window 13. By displaying thefirst contrast image in the area 22, it is possible to orient thecurrent video search image in such a way that parts of the tooth 10 and11 which are represented in the contrast image and in the search imageoverlap and are brought into congruence such that it is possible toattach the second recording created afterward.

In order to be able to bring the moving video search image intocongruence with the first contrast image, a semi-transparentrepresentation of the first image is preferred. The semi-transparentrepresentation can be realized in different ways, e.g., the imageinformation items of both recordings can be additively mixed. Suchmixing is represented in FIG. 5. Proceeding from the still image 31,stored in a memory, and the search image 32 currently recorded by thecamera, the image information items are added point by point by means ofa summer 33, and displayed as image 34. A further possibility consistsin the representation 34 of the image information items alternating rowby row or column by column, so that rows 1, 3, 5, 7, represent the imageinformation items of the still image 31, and rows 2, 4, 6, 8, representthe image information items of the search image 32. Such arepresentation 34 is referred to as intermeshing and takes place onlywhere both images yield information, that is to say in the area 22 ofthe window 13 (FIG. 3). In the area 21, where only one image, namely thecurrent video search image, yields information, the latter isexclusively represented. Instead of row-by-row intermeshing,point-by-point intermeshing can also be effected, by a procedure inwhich individual points of the images 31, 32 are contained alternatelyin the representation 34, thereby producing a type of chessboard pattern(FIG. 7).

By virtue of the possibility of stringing together a plurality ofrecordings and the transmission of the depth data measured, it ispossible to measure significantly larger areas beyond the image zone tobe measured by the camera and to use them as a model of the entireobject which is co-ordinated among the individual recordings.

1. A method for detecting and representing one or more objects using acamera, said method comprising the steps of: creating a first recording;producing a still image; inserting the still image into a current,moving search image in at least one sub-area, so that both images arerecognizable; positioning the camera in such a way that the search imageattains congruence with the inserted still image in said at least onesub-area; and initiating a second recording, wherein the recordingsyield a three-dimensional data record containing depth values, the firstand second recordings are correlated with one another computationallyusing the depth values of a common area, the still image is displacedand rotated by at least one of a predetermined distance and apredetermined angle, and is superposed on the moving search image onlyin said at least one sub-area, and the first and second recordings aredisplaced and rotated by at least 1/10 of the extent of the recording inone direction relative to one another.
 2. The method as claimed in claim1, wherein: the first and second recordings essentially relate to thesame object, the recordings being made from different directions.
 3. Themethod as claimed in claim 1, wherein: the object to be recorded in thesecond recording has been altered, the surroundings of the alteredobject essentially being unchanged.
 4. The method as claimed in claim 1,wherein: the displacement and rotation are effected in a mannerdependent on the area to be recessed using an anatomical knowledge aboutthe expected image information of the second recording.
 5. The method asclaimed in claim 1, wherein: the first recording is insertedsemi-transparently into the search image.
 6. The method as claimed inclaim 1, wherein: a semi-transparent superposition of the still imageand moving search image is produced by additive mixing.
 7. The method asclaimed in claim 1, wherein: the still image and the moving search imageare represented in such a way that the image information items alternaterow by row, or column by column or point by point.
 8. The method asclaimed in claim 5, wherein: the superposition of the image informationitems takes place only in the areas where both images yield informationitems, and at the locations where only one image yields information,this image is exclusively represented.
 9. The method as claimed in claim5, wherein: the still image is represented in one hue in thesuperposition, and the search image is represented in another hue. 10.The method as claimed in claim 5, further comprising the step of:modifying the still image by image-processing means for improvingrecognition accuracy of the still image.
 11. The method as claimed inclaim 9, wherein: the hue of the still image is red, and the hue of thesearch image is black or white.
 12. The method as claimed in claim 1,wherein: the objects include at least one of teeth, their preparationsand their immediate vicinity.